Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add simplistic exponential backoff retry #6

Merged

Conversation

batmat
Copy link
Contributor

@batmat batmat commented Sep 14, 2018

Introducing options.delay and options.factor.

Now, by default even without specifying options.delay, there's a default
delay of 100ms between retries in case of at least one failure.

I think it makes sense as a default behavior, and anyway users can
override this if needed.

Introducing options.delay and options.factor.

Now, by default even without specifying options.delay, there's a default
delay of 100ms between retries in case of at least one failure.

I think it makes sense as a default behavior, and anyway users can
override this if needed.
@batmat
Copy link
Contributor Author

batmat commented Sep 14, 2018

If the code suits your idea, I will add a few docs.

@coveralls
Copy link

coveralls commented Sep 14, 2018

Pull Request Test Coverage Report for Build 31

  • 18 of 18 (100.0%) changed or added relevant lines in 1 file are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.3%) to 98.611%

Totals Coverage Status
Change from base Build 22: 0.3%
Covered Lines: 48
Relevant Lines: 48

💛 - Coveralls

Copy link

@markstos markstos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think adding a default 100ms delay is a sound idea, and more likely to succeed than a 0ms delay.

I left a couple comments above for suggested improvements.

index.js Outdated Show resolved Hide resolved
logger.debug(`waiting for ${delay} ms before next retry for ${options.uri}`);
resolve(fetchDataWithRetry(tryCount));
}, delay);
});

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider multiplying the delay by the factor after the fetch.

On might expect the second request to happen 100 ms after the first request, but currently it's 100 ms * factor instead.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@markstos nope, this is in the catch clause. So as I understand it, this is already on the first failure. I did have the same thought as you are, and agree waiting on the first download would be wasteful :-).

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah sorry, I didn't read enough code context. Well done.

batmat added a commit to batmat/evergreen that referenced this pull request Sep 14, 2018
While a few PRs are in flight, we can move forward without needing to go
through an even heavier fork.

void666/request-promise-retry#6
@batmat
Copy link
Contributor Author

batmat commented Sep 14, 2018

Heh, go home @TravisCI, you're drunk if you're saying my second commit doesn't build if the first did 😋.
Does closing/reopening possibly retrigger a build? Possibly some flakiness?

@batmat batmat closed this Sep 14, 2018
@batmat batmat reopened this Sep 14, 2018
@void666
Copy link
Owner

void666 commented Sep 14, 2018

Build gets triggered for every push I presume. Will review and close the requests. Thanks @batmat

@batmat
Copy link
Contributor Author

batmat commented Sep 15, 2018

I realized my code does not work wrt. exponential backoffs. Only delays are taken in account 🤦‍♂️.

Additional commit on its way.

@batmat
Copy link
Contributor Author

batmat commented Sep 17, 2018

@void666 does this PR seem correct to you? Anything else you would like me to add?
Thanks!

@batmat
Copy link
Contributor Author

batmat commented Oct 1, 2018

@void666 gentle ping. Do you think you'll be able to take a look soon-ish? :-). Thanks

@void666
Copy link
Owner

void666 commented Oct 2, 2018 via email

@Legogris
Copy link

Legogris commented Nov 27, 2018

@void666 : Any chance we could get this merged soon? :)

@void666
Copy link
Owner

void666 commented Nov 27, 2018

@batmat @Legogris sincere apologies for the delay, saw the PR, changes looks good. @batmat Would be great if you can provide an update on the readme as well?

@mmcguff
Copy link

mmcguff commented Apr 30, 2019

    delay: 2000 // will delay retries by 2000 ms.  The default is 100. 
    factor: 2 // will multiple the delay by the factor each time a retry is attempted.  (delay * factor)

@batmat Just add this to the readme and it should be good. I don't totally understand how the factor will change over multiple retries but I think this change is important to get pushed soon as there are many use cases this would be helpful.

@batmat
Copy link
Contributor Author

batmat commented Apr 30, 2019

@mmcguff will try to add it later, but if you can use "GitHub suggestions", I will be able to add this directly from the phone or so (going to be a bit afk starting from now until tomorrow included, public holiday here).
Thanks!

@stegel
Copy link

stegel commented Oct 30, 2019

was this ever considered for merge @void666 @batmat
thank you

@markstos
Copy link

I recommend the the requestretry module instead. The design is very similar, but it has already shipped this feature. I'm unsubscribing from this thread. https://www.npmjs.com/package/requestretry

Copy link
Owner

@void666 void666 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Merging for next release.

@void666
Copy link
Owner

void666 commented Oct 31, 2019

@mmcguff will try to add it later, but if you can use "GitHub suggestions", I will be able to add this directly from the phone or so (going to be a bit afk starting from now until tomorrow included, public holiday here).
Thanks!

Updated the verbose, and resolved the conflicts.

@void666 void666 merged commit 622e641 into void666:master Oct 31, 2019
@batmat batmat deleted the add-retry-for-support-of-exp-backoff-and-more branch October 31, 2019 09:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants